Image capture control device, image capture control method, and recording medium

ABSTRACT

An image capture control device includes a recognition unit that determines whether a peripheral situation corresponds to a predetermined situation based on image data, and a controller that controls an infrared light irradiation unit to increase a pulse number of transmission pulses to be emitted to a target, when the recognition unit determines that the peripheral situation corresponds to the predetermined situation.

TECHNICAL FIELD

The present disclosure relates to an image capture control device, animage capture control method, a program, and a recording medium.

BACKGROUND ART

PTL 1 discloses a distance image sensor of a time-of-flight (TOF) typeas a sensor for capturing an image of a measuring object that moves orstands still (hereinafter, referred to as a “target”), and measuring adistance to the target.

Such a distance image sensor of the TOF type includes an infrared lightirradiation unit and an infrared light reception unit. A distancebetween the distance image sensor and the target is measured based on atime difference or a phase difference between irradiation timing atwhich the infrared light irradiation unit emits irradiation light andlight reception timing at which the infrared light reception unitreceives reflected light (that is, light in which the irradiation lightis reflected by the target).

CITATION LIST Patent Literature

PTL 1: Unexamined Japanese Patent Publication No. 59-79173

SUMMARY

The present disclosure provides a technique for accurately measuring adistance to a target.

An aspect of the present disclosure is directed to an image capturecontrol device that includes a recognition unit that determines whethera peripheral situation corresponds to a predetermined situation based onimage data, and a controller that controls an infrared light irradiationunit to increase a pulse number of transmission pulses to be emitted toa target when the recognition unit determines that the peripheralsituation corresponds to the predetermined situation.

Note that an aspect of the present disclosure may be directed to amethod, a program, and a tangible recording medium that records theprogram and is not transitory.

According to the present disclosure, a distance to a target can bemeasured with high accuracy.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a schematic diagram illustrating a configuration of an imagingdevice into which an image capture control device according to anexemplary embodiment of the present disclosure is embedded.

FIG. 2 is a schematic diagram illustrating a configuration of the imagecapture control device according to the exemplary embodiment of thepresent disclosure.

FIG. 3 is a schematic diagram illustrating a configuration of a TOF typedistance image sensor.

FIG. 4A is a schematic diagram illustrating states of irradiation lightand reflected light when an image capture control method of theexemplary embodiment is not performed.

FIG. 4B is a schematic diagram illustrating the states of theirradiation light and the reflected light when the image capture controlmethod of the exemplary embodiment is performed.

FIG. 4C is a schematic diagram illustrating a modified example of theirradiation light and the reflected light when the image capture controlmethod of the exemplary embodiment is performed.

FIG. 5A is a view displaying visible light image data of a blackvehicle.

FIG. 5B is a view displaying distance image data of the vehicleillustrated in FIG. 5A obtained when the image capture control method ofthe exemplary embodiment is not performed.

FIG. 5C is a view in which a contour extracted by a contour extractionunit is displayed on a distance image in FIG. 5B.

FIG. 5D is a view displaying the distance image data of the vehicleillustrated in FIG. 5A obtained when the image capture control method ofthe exemplary embodiment is performed.

FIG. 6 is a flowchart of a distance detection method.

FIG. 7 is a flowchart of a process for detecting a color of a target.

FIG. 8 is a schematic diagram of the vehicle when three corners thereofcan be viewed.

DESCRIPTION OF EMBODIMENT

Prior to describing an exemplary embodiment according to the presentdisclosure, a problem found in a conventional technique will briefly bedescribed. In the distance image sensor disclosed in, for example, PTL1, distance measuring accuracy may be lowered when a situation thatlowers light intensity of the reflected light to be received by aninfrared light reception unit is present in the periphery.

Exemplary Embodiment

With reference to FIGS. 1 to 8, an exemplary embodiment according to thepresent disclosure will be described below. Note that in the followingdescription of the exemplary embodiment, components (including elementalsteps) are not always essential, except for a case of clearly indicatingthat they are essential and a case in which it is conceivable that theyare obviously essential in terms of a principle, for example. Note thatinfrared light described in the following exemplary embodiment may benear-infrared light.

1 IMAGING DEVICE

With reference to FIG. 1, a schematic configuration of imaging device100 into which an image capture control device according to the presentdisclosure is embedded will be described first. FIG. 1 is a blockdiagram illustrating a configuration example of the imaging device.

Imaging device 100 illustrated in FIG. 1 generates image data such asvisible light image data that is an image used for identifying anexternal appearance of a target, infrared image data (hereinafter,referred to as “IR image data”), and distance image data of a distancebetween imaging device 100 and the target. The imaging device 100generates object identification (ID) data of the target.

Imaging device 100 described above is mounted on a vehicle as a devicethat images the periphery of the vehicle (typically, the front), forexample. The image data and the object ID data that are generated byimaging device 100 are transmitted to an electronic control unit (ECU)of an advanced driving assistant system (hereinafter, referred to as an“ADAS”) disposed at a subsequent stage of imaging device 100.

1.1 Specific Configuration of Imaging Device

A specific configuration of imaging device 100 illustrated in FIG. 1will be described below. Imaging device 100 includes infrared lightirradiation unit 110, infrared light reception unit 120, visible lightreception unit 130, and image capture control device 140.

1.2 Infrared Light Irradiation Unit

Infrared light irradiation unit 110 irradiates at least an imaging rangeof the distance image data with infrared light pulses (hereinafter,referred to as “transmission pulses”). Specifically, infrared lightirradiation unit 110 emits, for example, irradiation light 111 a, 111 bas illustrated in FIGS. 4A, 4B, respectively. Irradiation light 111 a,111 b will be described later.

Conditions of the transmission pulses (e.g., a width, amplitude, pulseintervals, a pulse number of pulses) emitted by infrared lightirradiation unit 110 are controlled by image capture control device 140described later.

1.3 Infrared Light Reception Unit

Infrared light reception unit 120 is, for example, a complementary metaloxide semiconductor (CMOS) image sensor, and receives the infrared lightto generate the IR image data.

Light reception conditions (e.g., an exposure time, exposure timing, andthe number of exposure times) of infrared light reception unit 120 arecontrolled by image capture control device 140 described later.

1.4 Visible Light Reception Unit

Visible light reception unit 130 is the CMOS image sensor, for example,and receives visible light of black and white (BW) or visible light ofcolor (red, green, and blue (RGB)) to generate visible light image data.

Light reception conditions (e.g., an exposure time, exposure timing, andthe number of exposure times) of visible light reception unit 130 arecontrolled by image capture control device 140 described later.

In the present exemplary embodiment, infrared light reception unit 120and visible light reception unit 130 are configured with common imagesensor 160. However, infrared light reception unit 120 and visible lightreception unit 130 can be configured with separated image sensors.

Further, in the present exemplary embodiment, an optical system (notillustrated) that introduces light (infrared light and visible light) toinfrared light reception unit 120 and visible light reception unit 130is a common optical system. However, separated optical systems mayintroduce light to infrared light reception unit 120 and visible lightreception unit 130.

1.5 Image Capture Control Device

Image capture control device 140 controls units configuring imagingdevice 100. Image capture control device 140 includes recognition unit141, controller 142, and output unit 150.

Image capture control device 140 is configured with, for example, inputterminal 140A, output terminal 140B, microprocessor 140C, program memory140D, and main memory 140E as illustrated in FIG. 2.

The above program memory retains program P1. The program memory may be anonvolatile semiconductor memory such as an electrically erasable andprogrammable read only memory (EEPROM).

The above main memory stores various pieces of data associated withexecution of a program. The main memory may be a volatile semiconductormemory such as a static random access memory (SRAM) or a dynamic randomaccess memory (DRAM).

The microprocessor reads the program from the program memory andexecutes the program using the main memory to implement variousfunctions of Image capture control device 140.

The functions of image capture control device 140 may be implemented asa logic circuit such as a field programmable gate array (FPGA) or anapplication specific integrated circuit (ASIC), or a program.

Hereinafter, the various functions of image capture control device 140will be described.

1.6 Recognition Unit

Recognition unit 141 determines whether a peripheral situationcorresponds to a predetermined situation based on the visible lightimage data generated by visible light reception unit 130. Recognitionunit 141 includes distance detector 143, contour extraction unit 144,color detector 145, backlight detector 146, fog detector 147, and objectextraction unit 148. Note that recognition unit 141 does not always needto include all configurations described above, but may include a part ofthe configurations.

The predetermined situation is a situation in which light intensity ofreflected light is lowered. The reflected light is light in whichirradiation light 111 a, 111 b of infrared light irradiation unit 110 isreflected by the target. Specifically, the predetermined situation is asituation in which backlight is present in the periphery, a situation inwhich fog is preset in the periphery, and a situation in which a colorwhose reflection rate for the IR light is lower than a predeterminedthreshold value for the reflection rate (e.g., black or gray:hereinafter referred to as a “low reflective color”) is included.

1.7 Distance Detector

Distance detector 143 detects a distance to the target based on the IRimage data generated by infrared light reception unit 120. Distancedetector 143 generates the distance image data based on the detecteddistance. A distance detection method of distance detector 143 will bedescribed later.

Distance detector 143 can extract a part of the IR image data byregionally dividing the IR image data. The distance image data isgenerated based on the extracted part of the IR image data. Note thatthe part of the IR image data includes the IR image data associated withthe target, for example.

1.8 Contour Extraction Unit

Contour extraction unit 144 extracts a contour of the target from thevisible light image data generated by visible light reception unit 130or the IR image data generated by infrared light reception unit 120, togenerate contour data. Note that, when contour extraction unit 144extracts the contour of the target from the IR image data to generatethe contour data, imaging device 100 may not include visible lightreception unit 130.

Specifically, when the visible light image data or the IR image data ofthe target is vehicle 201 illustrated in FIG. 5A, contour extractionunit 144 extracts contour 200 as illustrated in FIG. 5C from thisvisible light image data. Note that the visible light image data or theIR image data may simply be referred to as image data.

1.9 Color Detector

Color detector 145 determines whether the target includes the lowreflective color based on the visible light image data or the IR imagedata. Note that the predetermined threshold value for the reflectionrate is set to be a value associated with brightness, for example.

Specifically, color detector 145 determines whether a portion of the lowreflective color is present inside the contour of the target extractedby contour extraction unit 144 in the visible light image data or the IRimage data. Color detector 145 detects a ratio of an area of the portionof the low reflective color to an entire area inside the contour.

1.10 Backlight Detector

Backlight detector 146 determines whether the backlight is present basedon the visible light image data or the IR image data. Note that whetherthe backlight is present is determined based on information onbrightness of the visible light image data or the IR image data, forexample. Besides this method, a method for determining whether thebacklight is present can adopt various methods that have beenconventionally known.

1.11 Fog Detector

Fog detector 147 determines whether the fog is present based on thevisible light image data or the IR image data. Note that whether the fogis present is also determined based on the information on the brightnessof the visible light image data or the IR image data, for example.Besides this method, a method for determining whether the fog is presentcan adopt various methods that have been conventionally known.

1.12 Object Extraction Unit

Object extraction unit 148 extracts the target (that is, an object) fromthe distance image data generated by distance detector 143. Objectextraction unit 148 extracts distance image data of a portioncorresponding to the target in the distance image data generated bydistance detector 143. Note that, when imaging device 100 is mounted onthe vehicle, the target is various objects associated with vehicletraveling, such as another vehicle, a pedestrian, or a traffic sign.

Object extraction unit 148 further determines whether a portion whosedistance is inappropriately detected is present in the distance imagedata associated with the target.

When determining that the distance image data associated with the targetis inappropriate, object extraction unit 148 instructs contourextraction unit 144 to extract the contour of the target from thevisible light image data or the IR image data. Object extraction unit148 instructs contour extraction unit 144 directly or through controller142 to be described later.

When color detector 145 determines that the target includes the lowreflective color, object extraction unit 148 can instruct controller 142to increase or decrease a pulse number of transmission pulses ofinfrared light irradiation unit 110. Note that contour extraction unit144 may instruct controller 142 to increase or decrease the pulse numberof transmission pulses, instead of object extraction unit 148.

When instructing controller 142 to increase or decrease the pulse numberof transmission pulses, object extraction unit 148 or contour extractionunit 144 instructs controller 142 to decrease a frame rate of the IRimage data.

Object extraction unit 148 further generates object ID data of thetarget. Object extraction unit 148 then gives an object ID to the targetin the distance image data.

Note that recognition unit 141 outputs output data such as the object IDdata, the IR image data, the distance image data, and the visible lightimage data from the output terminal of output unit 150. The output datais transmitted to the ECU of the ADAS described above, for example. Notethat the output data includes at least one of the object ID data, the IRimage data, the distance image data, and the visible light image data.

1.13 Controller

Controller 142 controls, for example, a width and amplitude (intensity)of the transmission pulses emitted by infrared light irradiation unit110, pulse intervals, and the pulse number of transmission pulses.

Specifically, when recognition unit 141 determines that the peripheralsituation corresponds to the predetermined situation, controller 142controls infrared light irradiation unit 110 to increase or decrease thepulse number of transmission pulses.

Further, when recognition unit 141 determines that the target includesthe low reflective color (that is, corresponds to the predeterminedsituation), controller 142 controls infrared light irradiation unit 110to increase or decrease the pulse number of transmission pulses. Notethat, when a vehicle speed (or a relative speed between imaging device100 and the target) exceeds a predetermined threshold value for thespeed, the pulse number of transmission pulses may not be increased,which is a matter of course. Note that the vehicle speed is obtainedfrom the ECU of the vehicle, for example.

Controller 142 may control infrared light irradiation unit 110 toincrease or decrease intensity (luminosity) of the transmission pulsesas well as increase or decrease the pulse number of transmission pulses.

In addition, when a portion having a distance difference greater than orequal to a predetermined threshold value for a distance is presentinside the contour of the target in the distance image data, controller142 may control recognition unit 141 to determine whether the peripheralsituation corresponds to the predetermined situation based on thevisible light image data.

Controller 142 may control distance detector 143 to extract the part ofthe IR image data by increasing or decreasing the pulse number oftransmission pulses, and regionally dividing the IR image data generatedby infrared light reception unit 120.

In this case, controller 142 may control distance detector 143 togenerate the distance image data based on the IR image data of theextracted part. Note that the part of the IR image data includes the IRimage data associated with the target, for example.

Controller 142 controls an exposure time and exposure timing of infraredlight reception unit 120. Controller 142 also controls an exposure timeand exposure timing of visible light reception unit 130.

In the present exemplary embodiment, infrared light reception unit 120and visible light reception unit 130 are configured with the commonimage sensor. Therefore infrared light reception unit 120 and visiblelight reception unit 130 are synchronized with each other in exposuretime and exposure timing.

Note that infrared light reception unit 120 and visible light receptionunit 130 can be configured with separated image sensors. In this case,controller 142 controls infrared light reception unit 120 and visiblelight reception unit 130 such that the IR image data, the distance imagedata, and the visible light image data correspond to one another in aone-to-one basis.

Controller 142 also sets frame rates for the IR image data generated byinfrared light reception unit 120 and the visible light image datagenerated by visible light reception unit 130.

Specifically, controller 142 sets the frame rates for the IR image dataand the visible light image data according to the pulse number oftransmission pulses emitted by infrared light irradiation unit 110.

For example, when the pulse number of transmission pulses emitted byinfrared light irradiation unit 110 is greater than that in a normalstate, controller 142 sets the frame rate for the IR image data smallerthan that in the normal state.

When color detector 145 determines that an area rate of the portion withthe low reflective color to an entire area inside the contour of thetarget is less than or equal to a predetermined threshold value for thearea rate, controller 142 may not decrease the frame rate for the IRimage data.

When the target is a moving object, controller 142 may not decrease theframe rate for the IR image data.

Further, controller 142 may set the frame rate for the IR image dataaccording to the vehicle speed obtained from the ECU of the vehicle.

Specifically, when the vehicle speed (or the relative speed betweenimaging device 100 and the target) is greater than or equal to thepredetermined threshold value for the speed, controller 142 may notdecrease the frame rate for the IR image data.

1.14 Output Unit

Output unit 150 includes the output terminal. Output unit 150 outputs anoutput signal to, for example, the ECU of the ADAS from the outputterminal. Note that the output signal includes at least one type of dataamong the IR image data, the visible light image data, the distanceimage data, and the object ID data, for example.

2 IMAGING DATA GENERATED BY IMAGING DEVICE

Hereinafter, the visible light image data, the IR image data, thedistance image data, and the object ID data that are generated byimaging device 100 will be described.

2.1 Visible Light Image Data

The visible light image data is generated based on the visible lightreceived by visible light reception unit 130. The visible light imagedata is used when contour extraction unit 144 extracts the contour ofthe target.

The visible light image data is also used when backlight detector 146detects the backlight. The visible light image data is also used whenfog detector 147 detects the fog.

In the present exemplary embodiment, the visible light image data isgenerated so as to correspond to the IR image data generated by infraredlight reception unit 120 in the one-to-one basis. Accordingly a positionof the target in the visible light image data and a position of thetarget in the IR image data correspond to each other in the one-to-onebasis. Note that the visible light image data is output from the outputterminal of output unit 150 as a part of the output data.

2.2 IR Image Data

The IR image data is generated based on the IR light received byinfrared light reception unit 120. The IR image data is transmitted todistance detector 143 and used for generation of the distance imagedata. The IR image data is output from the output terminal of outputunit 150 as a part of the output data. Further, the IR image data istransmitted to contour extraction unit 144 and used for the contourextraction.

2.3 Distance Image Data

The distance image data is generated based on the distance data to thetarget detected by distance detector 143. The distance image data isgenerated as data including coordinate information and distanceinformation. The distance image data is output from the output terminalof output unit 150 as a part of the output data.

2.4 Object ID Data

The object ID data is an identifier given to an object (that is, target)that is extracted by object extraction unit 148 from the distance imagedata. The object ID data is output from the output terminal of outputunit 150 as a part of the output data.

Note that data generated by imaging device 100 is stored in a recordingmedium such as a memory (not illustrated) included in imaging device100. In this case, output unit 150 may output the above data from therecording medium at appropriate timing.

3 DISTANCE DETECTION METHOD

Hereinafter, with reference to FIGS. 1, 3, 6, 7, a method for measuringthe distance to the target by imaging device 100 illustrated in FIG. 1will be described.

3.1 Distance Image Sensor

With reference to FIGS. 1, 3, a configuration of the distance imagesensor included in imaging device 100 will briefly be described first.

In imaging device 100 illustrated in FIG. 1, measurement of the distanceto the target is performed by the TOF type distance image sensor. Thedistance image sensor is configured with infrared light irradiation unit110, infrared light reception unit 120, and distance detector 143 thatare described above.

As illustrated in FIG. 3, the TOF type distance image sensor measuresdistance Z to the target based on a time difference or a phasedifference between irradiation timing of irradiation light 111 a, 111 bemitted by infrared light irradiation unit 110 and reception timing, atinfrared light reception unit 120, of reflected light 112 a, 112 b thatis light in which irradiation light 111 a, 111 b is reflected by thetarget (in FIG. 3, a person), respectively. Note that the distance imagesensor can adopt various types of the TOF sensor.

3.2 Distance Detection Method

Next, a distance detection method that is performed using imaging device100 illustrated in FIG. 1 will be described with reference to FIGS. 6,7. Note that FIG. 6 is a flowchart of the distance detection method.FIG. 7 is a flowchart of a color detection process.

Note that in the following description, a case where the target is blackvehicle 201 as illustrated in FIG. 5A will be described. In FIG. 5A, adark portion is attached with a slanted lattice.

In an image capture control method of the present exemplary embodiment,the IR image data is first generated based on IR light received byinfrared light reception unit 120 in step 10 of FIG. 6.

Next, in step 11, distance detector 143 generates the distance imagedata based on distance data detected from the IR image data. Note thatthe distance detection method will be described later.

Next, in step 12, object extraction unit 148 extracts the target fromthe distance image data.

Next, in step 13, object extraction unit 148 generates the object ID forthe object and gives an object ID to the object.

Next, in step 14, object extraction unit 148 determines whether anon-detection portion is present in the distance image data of theobject. Note that the non-detection portion is a portion whose distanceis not detected accurately.

A case where the non-detection portion is present in the distance imagedata of the target (that is, the object) will be described withreference to FIGS. 5A to 5D. Note that FIG. 5A is a view displaying thevisible light image data of the black vehicle. FIG. 5B is a viewdisplaying the distance image data of the vehicle illustrated in FIG. 5Aobtained when the image capture control method of the present exemplaryembodiment is not performed.

FIG. 5C is a view displaying the contour of the object extracted fromthe visible light image data by the contour extraction unit whilesuperimposing on the distance image of FIG. 5B. FIG. 5D is a viewdisplaying the distance image data of the vehicle illustrated in FIG. 5Aobtained when image capture control method of the present exemplaryembodiment is performed.

Note that the case where the non-detection portion is present in thedistance image data of the object is exemplified by a case where thedistance image data of vehicle 201 illustrated in FIG. 5A is in a stateillustrated in FIG. 5B.

In other words, when an accurate distance is detected, the distanceimage data of vehicle 201 illustrated in FIG. 5A is in a stateillustrated in FIG. 5D. However, in the distance image data illustratedin FIG. 5B, the distance image data of a portion (that is, dark portion)other than head lights, a license plate, and a front grill of vehicle201 is not generated appropriately.

In other words, the distance of the dark portion of vehicle 201illustrated in FIG. 5A is not detected accurately. In this case, aportion corresponding to the dark portion (slanted lattice portion) ofvehicle 201 illustrated in FIG. 5A in the distance image dataillustrated in FIG. 5B corresponds to the non-detection portion.

In step 14, when “the non-detection portion being not present” isdetermined (NO in step 14), the distance detection process in this timeis terminated without performing the image capture control method of thepresent exemplary embodiment.

On the other hand, in step 14, when “the non-detection portion beingpresent” is determined (YES in step 14), recognition unit 141 detectswhether the peripheral situation corresponds to the predeterminedsituation in step 15.

Specifically, in step 15, recognition unit 141 determines whether thebacklight, the fog, or the low reflective color is present based on thevisible light image data generated by visible light reception unit 130.Note that the backlight is detected by backlight detector 146. The fogis detected by fog detector 147. Further, the low reflective color isdetected by color detector 145.

Hereinafter, in step 15, a method for detecting a color of the target bycolor detector 145 will be described.

3.3 Method for Detecting Low Reflective Color

FIG. 7 is the flowchart of the color detection process.

The following process is performed when recognition unit 141 determinesthat the non-detection portion is present in the distance image data ofthe target in step 14 of FIG. 6. The following describes a case wherethe target is vehicle 201 in FIG. 5A. However the target is not limitedto the vehicle.

First, in step 150, object extraction unit 148 instructs contourextraction unit 144 to extract the contour of the target. Specifically,the target is, for example, an article including the head lights, thelicense plate, and the front grill (that is, vehicle 201 in FIG. 5A) inthe distance image data illustrated in FIG. 5B.

Next, in step 151, contour extraction unit 144 extracts the contour ofthe target from the visible light image data. Specifically, contourextraction unit 144 extracts contour 200 of vehicle 201 from the visiblelight image data illustrated in FIG. 5A.

Next, in step 152, color detector 145 detects the low reflective colorinside the contour of the target extracted by contour extraction unit144 in the visible light image data. Specifically, color detector 145detects a black or gray color present inside contour 200 of vehicle 201in the visible light image data illustrated in FIG. 5A. The colordetection process is then terminated, and the process returns to step 15of FIG. 6.

Note that, even when the low reflective color is detected inside thecontour of the target in the visible light image data, if an area of alow reflective color portion is less than or equal to a predeterminedthreshold value for the area, it may be determined that the lowreflective color is not detected.

Furthermore, for example, as in vehicle 201 a illustrated in FIG. 8,whether the low reflective color is present may be determined based oncolors of three corners 202 a, 202 b, 203 a that can be detected in thevisible light image data among two corners (that is, nooks) in the frontend portion and both ends in a width direction of vehicle 201 a and twocorners (that is, nooks) in the rear end portion and both ends in thewidth direction of vehicle 201 a.

Next, in step 16, recognition unit 141 determines whether thepredetermined situation is detected (that is, whether to correspond tothe predetermined situation). In step 16, when it is determined that thesituation does not correspond to the predetermined situation (NO in step16), the distance detection process in this time is terminated withoutperforming the image capture control method of the present exemplaryembodiment.

Note that the case of not corresponding to the predetermined situationis a situation where the backlight and the fog are not detected from thevisible light image data, and the low reflective color is not detectedfrom the target in the visible light image data.

On the other hand, in step 16, when it is determined that the situationcorresponds to the predetermined situation (YES in step 15), the processproceeds to step 17.

Next, in step 17, recognition unit 141 instructs controller 142 toincrease the pulse number of transmission pulses of infrared lightirradiation unit 110.

Next, in step 18, controller 142 controls infrared light irradiationunit 110 to increase the pulse number of transmission pulses more thanthe pulse number of transmission pulses in the normal state. At the sametime, controller 142 controls the exposure time and the exposure timingfor infrared light reception unit 120 and visible light reception unit130 according to the pulse number of transmission pulses emitted byinfrared light irradiation unit 110.

Furthermore, controller 142 may set the frame rates for the IR imagedata and the visible light image data as described above, according tothe pulse number of transmission pulses emitted by infrared lightirradiation unit 110.

Next, in step 19, based on IR light received in a state where the pulsenumber of transmission pulses emitted by infrared light irradiation unit110 is increased (hereinafter, referred to as a “controlled state”),infrared light reception unit 120 generates IR image data (hereinafter,referred to as “IR image data in a controlled state”).

Next, in step 20, based on distance data detected from the IR image datain the controlled state, distance detector 143 generates distance imagedata (hereinafter, referred to as “distance image data in a controlledstate”). Controller 142 then returns the process to step 14.

Note that the distance image data in the controlled state does not needto be generated for all IR image data in the controlled state. In otherwords, based on the function included in distance detector 143, distancedetector 143 regionally divides the IR image data in the controlledstate, and extracts a part of the IR image data including data of thetarget. Distance detector 143 may generate the distance image data basedon the part of the IR image data. When only the part of the distanceimage data is processed in this manner, a processing load of distancedetector 143 can be reduced, whereby a lowering amount of the frame ratecan be reduced.

Note that, after the process returns to step 14 in the controlled state,when the non-detection portion is not present in the distance image dataof the target in the distance image data in the controlled state,controller 142 controls infrared light irradiation unit 110 to put backthe pulse number of transmission pulses to the pulse number oftransmission pulses in the normal state. The distance detection processin this time is terminated.

Note that in the present exemplary embodiment, the case where thecontour of the target is extracted from the visible light image data hasbeen described. However, for example, when the distance image based onthe IR image data is in the situation illustrated in FIG. 5B, it isobvious that the similar effect can be expected by collating thedistance image with a pattern stored in an internal table to determinethat this distance image is of an object with the low reflective color,and increasing the pulse number of transmission pulses through similarprocedures.

4 DISTANCE DETECTION METHOD AND ACTIONS AND EFFECTS OF THE PRESENTEXEMPLARY EMBODIMENT

Subsequently, with reference to FIGS. 4A, 4B, the distance detectionmethod and actions and effects of the present exemplary embodiment willbe described below. In the following description, an example of a methodfor detecting a distance with a distance image sensor of a so-calledindirect TOF type will be described.

Note that, when an image capture control method according to the presentexemplary embodiment is performed, the indirect TOF type or a direct TOFtype can be adopted as the distance image sensor. Further, with theimage capture control method according to the present exemplaryembodiment, similar actions and effects can be obtained regardless ofthe type of the distance image sensor.

4.1 Distance Detection Method

First, a method for detecting the distance to the target in a statewhere controller 142 does not control infrared light irradiation unit110 to increase the pulse number of transmission pulses (hereinafter,referred to as a “normal state”) will be described with reference toFIG. 4A.

In the normal state, irradiation light 111 a of infrared lightirradiation unit 110 includes first pulse 113 that is emitted first andsecond pulse 114 that is emitted with pulse interval L₁ spaced fromfirst pulse 113, as illustrated in FIG. 4A. First pulse 113 and secondpulse 114 are equal to each other in amplitude H₁ and width T_(p).

Note that, in the normal state, controller 142 controls infrared lightirradiation unit 110 to emit transmission pulses of a number having amargin with respect to an upper limit of irradiation capability.

For convenience of the description, in FIG. 4A, only a single pulse setincluding first pulse 113 and second pulse 114 is illustrated. Howeverthe irradiation light can be configured with a plurality of pulse sets.

Controller 142 controls infrared light reception unit 120 to performexposure at timing synchronized with that of first pulse 113 and secondpulse 114. In the present exemplary embodiment, as illustrated in FIG.4A, infrared light reception unit 120 performs three times of exposurethat are exposure S₀, exposure S₁, and exposure BG with respect toreflected light 112 a that is light in which irradiation light 111 a isreflected by the target.

Specifically, in exposure S₀, the exposure is started simultaneouslywith start of irradiation with first pulse 113 (that is, the rise offirst pulse 113), and is terminated after performing the exposure forexposure time T₀ that is preset according to a relationship withirradiation light 111 a. Exposure S₀ is exposure for receiving reflectedlight of first pulse 113.

Note that light reception data (that is, light amount information) D₀with exposure S₀ includes reflected light component S₀ of first pulse113 attached with a slanted lattice and background component BG attachedwith a satin pattern in FIG. 4A. Reflected light 112 a is attenuated soas to be smaller than irradiation light 111 a, so that amplitude H₂ ofreflected light component S₀ is smaller than amplitude H₁ of first pulse113.

Herein, time difference Δt is present between the rise of first pulse113 and the rise of reflected light component S₀ of first pulse 113. Inother words, reflected light component S₀ of first pulse 113 rises aftertime difference Δt elapses from the rise of first pulse 113.

Time difference Δt is a time that is required by light to reciprocate aroute with distance Z (refer to FIG. 3) from imaging device 100 to thetarget.

In exposure S₁, the exposure is started simultaneously with terminationof irradiation with second pulse 114 (that is, the fall of second pulse114), and is terminated after performing the exposure for exposure timeT₀ similar to exposure S₀. Exposure S₁ is exposure for receiving thereflected light of second pulse 114.

Light reception data D₁ with exposure S₁ includes component S₁ (aslanted lattice portion in FIG. 4A) of a part of the reflected lightcomponent of second pulse 114 and background component BG attached withthe satin pattern in FIG. 4A.

Note that component S₁ of the part of the reflected light component canbe expressed by Expression (1) below.

S ₁ =S ₀×(Δt/T _(p))  [Expression 1]

In exposure BG, the exposure is started at timing excluding reflectedlight components of first pulse 113 and second pulse 114, and isterminated after performing the exposure for exposure time T₀ similar toexposure S₀ and exposure S₁.

Exposure BG is exposure for receiving only an infrared light componentin external light (that is, a background component). Accordingly, lightreception data D_(BG) with exposure BG includes only backgroundcomponent BG indicated with the satin pattern in FIG. 4A.

From a relationship between irradiation light 111 a and reflected light112 a as described above, distance Z from imaging device 100 to thetarget can be calculated with Expressions (2) to (4) below. Here “T_(P)”is a width of each of first pulse 113 and second pulse 114, and “c” is alight speed. Note that D_(BG) in the following expressions is lightreception data generated with exposure BG described above.

S ₀ =D ₀ −D _(BG)  [Expression 2]

S ₁ =D ₁ −D _(BG)  [Expression 3]

$\begin{matrix}\begin{matrix}{Z = {c \times \left( {\Delta \; {t/2}} \right)}} \\{= {\left\{ {\left( {c \times T_{p}} \right)/2} \right\} \times \left( {\Delta \; {t/T_{p}}} \right)}} \\{= {\left\{ {\left( {c \times T_{p}} \right)/2} \right\} \times \left( {S_{1}/S_{0}} \right)}}\end{matrix} & \left\lbrack {{Expression}\mspace{14mu} 4} \right\rbrack\end{matrix}$

When distance Z is detected with the above-described method, with smallamplitude of each of the reflected light components of first pulse 113and second pulse 114 (that is, with lowered intensity), signal-to-noise(SN) ratios of light reception data D₀ and light reception data D₁ aredecreased, whereby detection accuracy of distance Z may be lowered.

Hence, with the image capture control method according to the presentexemplary embodiment, when a situation in which light intensity of eachof the reflected light components of first pulse 113 and second pulse114 is lowered (that is, the predetermined situation) is present,controller 142 controls infrared light irradiation unit 110 to increasethe pulse number of transmission pulses.

Hereinafter, with reference to FIG. 4B, a method for detecting thedistance to the target in the controlled state will be described.

In the following description, an example in which a pulse number oftransmission pulses per frame is increased to twice the pulse number oftransmission pulses in the normal state will be described. Note that thedescription of an overlapping portion with the description in the normalstate will be omitted.

In the controlled state, irradiation light 111 b of infrared lightirradiation unit 110 is configured with pulse sets twice those in thenormal state as illustrated in FIG. 4B. In other words, irradiationlight 111 b in the controlled state includes two pulse sets each ofwhich is similar to the pulse set in the normal state illustrated inFIG. 4A. Amplitude H₁ and width T_(P) of each of first pulse 113 andsecond pulse 114 configuring the pulse set are similar to those in thenormal state.

In the controlled state, two pulse sets each of which includes firstpulse 113 and second pulse 114 configure one frame of the distance imagedata. Consequently, controller 142 causes the frame rates for thevisible light image data generated by visible light reception unit 130and the IR image data generated by infrared light reception unit 120that has received reflected light 112 b to be lower than the frame ratesin the normal state.

Note that, for convenience of the description, the pulse number oftransmission pulses of irradiation light 111 b illustrated in FIG. 4B isincreased to be twice that of irradiation light 111 a illustrated inFIG. 4A. Note that the irradiation light in the controlled state is notlimited to the case in FIG. 4B.

Exposure timing of infrared light reception unit 120 is similar to thecase illustrated in FIG. 4A. In other words, infrared light receptionunit 120 performs three times of exposure that are exposure S₀, exposureS₁, and exposure BG with respect to each of two pulse sets configuringirradiation light 111 b.

In particular, in the controlled state, reflected light component S₀ offirst pulse 113 in the first pulse set and reflected light component S₀of first pulse 113 in the second pulse set are added to each other. Notethat reflected light component S₀ is calculated with above Expression(2).

On the other hand, component S₁ of a part of a reflected light componentof second pulse 114 in the first pulse set and component S₁ of a part ofa reflected light component of second pulse 114 in the second pulse setare added to each other. Note that component S₁ of the part of thereflected light component is calculated with above Expression (3).

The added values are substituted into above Expression (4) to detectdistance Z from imaging device 100 to the target. Note that, uponperforming the addition described above, white noise is reduced, therebysuppressing an influence of the white noise on detection accuracy ofdistance Z.

4.2 Actions and Effects of the Present Exemplary Embodiment

As described above, according to the present exemplary embodiment, evenwhen backlight or fog is present in the periphery or the target has thelow reflected light reflective color (that is, when the peripheralsituation corresponds to the predetermined situation), the distance fromimaging device 100 to the target can be detected with high accuracy. Asa result distance image data accurately reflecting the distance to thetarget can be generated.

In other words, in the present exemplary embodiment, when the peripheralsituation corresponds to the predetermined situation, infrared lightirradiation unit 110 is controlled to increase the pulse number oftransmission pulses.

Specifically, in the normal state in which the image capture controlmethod of the present exemplary embodiment is not performed, whenintensity of each of the reflected light components of first pulse 113and second pulse 114 is lowered (that is, when the SN ratio is lowered),infrared light irradiation unit 110 is controlled such that irradiationlight 111 b whose transmission pulses are increased more than those ofirradiation light 111 a in the normal state is emitted.

Then reflected light components S₀ of reflected light 112 b are added toeach other, and components S₁ of parts of the reflected light componentsof reflected light 112 b are added to each other. As a result, the SNratio of data used for detection of the distance is increased, wherebydistance Z can be detected with high accuracy.

Accordingly, as illustrated in FIG. 5B, even when the non-detectionportion is present in the distance image data of the target, distance Zcan be detected with high accuracy by performing the image capturecontrol method of the present exemplary embodiment. This can generatethe distance image data accurately reflecting the distance to the targetas illustrated in FIG. 5D.

Note that, for convenience of the description, in FIG. 5D, portion 300with a shorter distance is attached with a slanted lattice, and portion301 with a longer distance is attached with a satin pattern.

5 APPENDIX

In steps 17, 18 of the flowchart in FIG. 6, controller 142 may decreasethe pulse number of transmission pulses as a condition for thetransmission pulses emitted by infrared light irradiation unit 110.Specifically, when the non-detection portion is present in step 14 andwhen the reflection rate of light received by infrared light receptionunit 120 is higher than a predetermined value or when brightness data issaturated, such as light reflected by a reflection plate, in steps 15,16, controller 142 may instruct infrared light irradiation unit 110 todecrease the pulse number of transmission pulses in step 17, andinfrared light reception unit 110 may be controlled to decrease thepulse number of transmission pulses in step 18.

As another example, in the controlled state, controller 142 can controlinfrared light reception unit 120 to perform the exposure at timingillustrated in FIG. 4C. Specifically, controller 142 controls infraredlight reception unit 120 to omit second exposure BG among exposuresillustrated in FIG. 4B.

Configurations of recognition unit 141 and controller 142 in imagecapture control device 140 may be implemented by a computer program. Thecomputer program may be provided while being stored in a recordingmedium such as a digital versatile disc (DVD), or may be stored in arecording medium such as a server device on a network, which can bedownloaded through the network.

Alternatively, recognition unit 141 and controller 142 in image capturecontrol device 140 can be implemented as physical circuits such aslarge-scale integration (LSI).

INDUSTRIAL APPLICABILITY

An image capture control device, an image capture control method, aprogram, and a recording medium according to the present disclosure aresuitable for an imaging device mounted on a vehicle, for example.

REFERENCE MARKS IN THE DRAWINGS

-   -   100: imaging device    -   110: infrared light irradiation unit    -   111 a, 111 b: irradiation light    -   112 a, 112 b: reflected light    -   113: first pulse    -   114: second pulse    -   120: infrared light reception unit    -   130: visible light reception unit    -   140: image capture control device    -   141: recognition unit    -   142: controller    -   143: distance detector    -   144: contour extraction unit    -   145: color detector    -   146: backlight detector    -   147: fog detector    -   148: object extraction unit    -   150: output unit    -   160: image sensor    -   201, 201 a: vehicle    -   200: contour    -   202 a, 202 b: corner    -   203 a: corner    -   300: portion with shorter distance    -   301: portion with longer distance

1. An image capture control device that controls an imaging device, theimaging device including an infrared light irradiation unit, an infraredlight reception unit, and a visible light reception unit, the imagecapture control device comprising: a distance detector that measures adistance to a target that reflects transmission pulses with atime-of-flight (TOF) method, based on a time difference between a timewhen the infrared light irradiation unit emits the transmission pulsesand a time when the infrared light reception unit receives infraredlight; a recognition unit that determines whether a peripheral situationof peripheral of the imaging device corresponds to a predeterminedsituation based on image data generated by the visible light receptionunit that has received visible light; and a controller that causes theinfrared light irradiation unit to increase a pulse number of thetransmission pulses to be emitted to the target when the recognitionunit determines that the peripheral situation corresponds to thepredetermined situation.
 2. The image capture control device accordingto claim 1, wherein the predetermined situation is a situation in whichthe target has a low reflective color that is a color whose reflectionrate to the transmission pulses is less than a predetermined thresholdvalue for the reflection rate.
 3. The image capture control deviceaccording to claim 2, wherein the distance detector generates distanceimage data based on infrared image data generated by the distancedetector, and the recognition unit extracts the target from the distanceimage data, and gives an object identification (ID) to the target. 4.The image capture control device according to claim 3, wherein therecognition unit extracts a contour of the target from the image data,and determines whether the low reflective color is present inside thecontour.
 5. The image capture control device according to claim 4,wherein the recognition unit determines whether a portion having adistance difference greater than or equal to a predetermined value in aportion corresponding to an inner side of the contour in the distanceimage data, and, when the recognition unit determines that the portionhaving the distance difference is present, determines whether aperipheral situation corresponds to the predetermined situation based onthe image data.
 6. The image capture control device according to claim2, wherein the controller causes a frame rate of infrared image datagenerated by the distance detector to be lower than a frame rate beforethe controller causes the infrared light irradiation unit to increasethe number according to the pulse number of the transmission pulses. 7.The image capture control device according to claim 6, wherein thecontroller determines whether an area rate of a portion with the lowreflective color in the target with respect to a whole of the target inthe image data is less than or equal to a predetermined threshold valuefor the area rate, and does not decrease the frame rate of the infraredimage data when determining that the area rate is less than or equal tothe predetermined threshold value for the area rate.
 8. The imagecapture control device according to claim 6, wherein the controllerdetermines whether the target is a moving object, and does not decreasethe frame rate of the infrared image data when determining that thetarget is the moving object.
 9. The image capture control deviceaccording to claim 2, wherein the controller controls the infrared lightirradiation unit to increase the pulse number of the transmissionpulses, and increase pulse intensity of the transmission pulses.
 10. Theimage capture control device according to claim 2, wherein therecognition unit extracts a part of infrared image data, the part beingof the target and regionally divided from the infrared image datagenerated by the distance detector, and generates distance image databased on the part of the infrared image data.
 11. The image capturecontrol device according to claim 2, wherein the controller that causesthe infrared light irradiation unit to increase a pulse number of thetransmission pulses to be emitted to the target when the recognitionunit determines that the target includes the low reflective color, and arelative speed with the target is less than or equal to a predeterminedthreshold value for the relative speed.
 12. The image capture controldevice according to claim 1, wherein the predetermined situation is asituation where backlight is present in the periphery.
 13. The imagecapture control device according to claim 1, wherein the predeterminedsituation is a situation where fog is present in the periphery.
 14. Animage capture control method that controls an imaging device includingan infrared light irradiation unit, an infrared light reception unit,and a visible light reception unit, the image capture control methodcomprising: measuring a distance to a target that reflects transmissionpulses with a time-of-flight (TOF) method, based on a time differencebetween a time when the infrared light irradiation unit emits thetransmission pulses and a time when the infrared light reception unitreceives infrared light; determining whether a peripheral situationcorresponds to a predetermined situation of the peripheral of imagingdevice based on image data generated by the visible light reception unitthat has received visible light; and controlling the infrared lightirradiation unit to increase a pulse number of the transmission pulsesto be emitted to the target when the peripheral situation is determinedto correspond to the predetermined situation.
 15. A recording mediumthat records a program of an image capture control method that controlsan imaging device including an infrared light irradiation unit, aninfrared light reception unit, and a visible light reception unit, theprogram causing a computer to perform: measuring a distance to a targetthat reflects transmission pulses with a time-of-flight (TOF) method,based on a time difference between a time when the infrared lightirradiation unit emits the transmission pulses and a time when theinfrared light reception unit receives infrared light; determiningwhether a peripheral situation corresponds to a predetermined situationbased on image data generated by the visible light reception unit thathas received visible light; and controlling the infrared lightirradiation unit to increase a pulse number of the transmission pulsesto be emitted to the target when the peripheral situation is determinedto correspond to the predetermined situation.